Neurons in Orbitofrontal Cortex Encode Economic Value

Economic choice is the behavior observed when individuals select one of multiple options available in alternative to each other. In economic choice, there is no intrinsically “correct” answer; the choice depends on subjective preferences. This behavior is traditionally the object of economic analysis1 and is also of primary interest in psychology2. However, the underlying mental processes and neuronal mechanisms are not well understood. Theories of human and animal choice have a cornerstone in the concept of value1–3. Consider for example a monkey offered one raisin versus one piece of apple. Behavioral evidence suggests that the animal chooses by assigning values to the two options4. But where and how are values represented in the brain? Here we show that, during economic choice, neurons in orbitofrontal cortex5–18 (OFC) encode the value of offered and chosen goods. Importantly, OFC neurons encode value independently of visuospatial factors and motor responses. For example, if a monkey chooses between A and B, neurons in OFC encode the value of the two goods independently of whether A is presented on the right and B on the left, or vice versa. This trait distinguishes OFC from other brain areas where value modulates activity related to sensory or motor processes19–25. Our results have broad implications for possible psychological models of economic choice. They suggest that economic choice is at its essence choice between goods, as opposed to choice between actions. In this scheme, neurons in OFC seem a good candidate network for value assignment underlying economic choice. In our experiments, monkeys choose between two types of juice (A and B; A preferred) offered in different amounts. For example, in the session shown in figure 1, the monkey chooses between water (juice A) and unsweetened Kool-Aid (juice B). Offer types include 1B:2A, 1B: 1A, 2B:1A, 3B:1A, 4B:1A, 6B:1A, and 10B:1A, and the “forced choices” 0B:1A and 3B:0A. Behaviorally, we observe a trade-off between juice type and juice quantity. The monkey chooses A when 1B, 2B, or 3B are available in alternative, it is roughly indifferent between the two juices when offered 4B:1A, and it chooses B when 6B or 10 B are available. We interpret this pattern of choice in terms of relative value of the two juices4: in this case, the value of 1A is roughly equal to the value of 4B. Fitting a sigmoid provides the better estimate V(1A)=V(4.1B). Assuming a linear value function, we obtain V(A)=4.1V(B). This equation puts different quantities of juices A and B on a same value scale. On this basis, we can compute for each trial the value of the juice chosen by the monkey. Expressing values in units of V(B), the chosen value is ≈4 when the monkey chooses 1A or 4B. When the monkey chooses 2A, the chosen value is ≈8. When the monkey chooses 6B, 10B, and 3B, the chosen value is respectively equal to 6, 10, and 3. Hence, we can make specific hypotheses regarding the Correspondence to: Camillo Padoa-Schioppa, Department of Neurobiology, Harvard Medical School 220, Longwood Avenue, Boston, MA, 02115, Tel: 617.4322805, Fax: 617.7347557, Email: [email protected]. Author Contributions C.P.S. designed the experiment, collected and analyzed the data, wrote the manuscript, and performed all aspects of the study. J.A.A. assisted for experimental design, data analysis and manuscript preparation. Author Information Reprints and permissions information is available at npg.nature.com/reprints and permissions. The authors declare no competing financial interest. Correspondence and request for materials should be addressed to C.P.S. ([email protected]). NIH Public Access Author Manuscript Nature. Author manuscript; available in PMC 2009 January 23. Published in final edited form as: Nature. 2006 May 11; 441(7090): 223–226. doi:10.1038/nature04676. N IH PA Athor M anscript N IH PA Athor M anscript N IH PA Athor M anscript neuronal representation of juice values. In different sessions and with different juices, we record different behavioral choice patterns. We then analyze each cell in relation to the choice pattern recorded in the same session. Our recordings focused on area 13 in OFC. Figure 2 illustrates the activity of one representative neuron. The cell’s activity does not depend on whether juice A is offered on the left or on the right (figure 2b). It also does not depend on whether the monkey chooses the juice on the left or the juice on the right (i.e., makes an eye movement to the left or to the right; figure 2c). However, the cell’s activity varies with the offer type. This is consistent across the neuronal population. We recorded the activity of 931 cells and we analyze their neuronal responses in seven time windows. We test the activity of each cell in each time window with a 3-way ANOVA (factors: [position of juice A] × [movement direction] × [offer type], p<0.001). Rarely do responses depend on either the spatial configuration of the offers or the motor output (<5% neurons). In contrast, the activity of 505 (54%) neurons varies significantly depending on the offer type in at least one time window. Pooling time windows, a total of 1379 responses are significantly modulated by the offer type (Supplementary Figure S2). The cell shown in figure 2d has a “U-shaped” response similar to that hypothesized for a neuron encoding the chosen value. For this session V(A)=1.9V(B). Accordingly, the activity of the cell is lower when the monkey chooses 1A or 2B (in units of V(B), chosen value ≈ 2), it is higher when the monkey chooses 2A or 4B (chosen value ≈ 4), and it is highest when the monkey chooses 3A or 6B (chosen value ≈ 6). A linear regression of this response on the variable chosen value provides R2=0.86. Similar U-shaped responses are frequent in OFC, and figures 3a–c illustrate three more examples. We also find other types of responses. For example, neuronal responses often reflect the value of one of the two juices alone. Figures 3d–e show two cells whose activity co-varies with the value of A offered and the value of B offered, respectively. We label these responses as related to the offer value. Other frequently observed responses vary in a binary fashion depending on the type of juice chosen by the monkey, independently of the amount (figure 3f). We interpret these responses as related to juice taste. While many neurons appear to encode chosen value, offer value or juice taste, the relation between their activity and these three variables could be subordinate to a correlation with other behavioral variables. For example, neurons in OFC might encode the number of squares on the monitor (or variables proportional to number, such as juice quantity, or absolute luminance of the visual stimulus). To cast a wide net, we examine the linear dependence of neuronal data on 19 possible variables (Supplementary Figure S1). For example, we analyze the variables chosen number, total number, and total value. We include in this analysis 1379 responses significantly modulated by the offer type, and we regress each response separately on each variable. Collectively, the 19 variables explain 1227 (89%) neuronal responses. However, the 19 variables are often highly correlated. To identify a few variables that best describe the neuronal population, we adapt procedures for variable selection commonly used in multi-linear regression in the presence of multi-collinearity. Both the stepwise and the best-subset methods identify variables offer value, chosen value, and taste, which explain well the large majority of responses (1085/1379=79% responses explained, mean R2=0.63). A post-hoc analysis indicates that the explanatory power of these three variables is significantly higher than that of challenging alternatives. Furthermore, data from the two monkeys analyzed separately provide statistically indistinguishable results. Finally, a bi-linear regression analysis indicates that in 890/1085 (82%) cases adding a second variable or a quadratic value term does not improve the regression significantly (Supplementary Results, p.S5–S10; Figures S4–S11). We conclude that, indeed, OFC responses encode variables offer value, chosen value, and taste. Padoa-Schioppa and Assad Page 2 Nature. Author manuscript; available in PMC 2009 January 23. N IH PA Athor M anscript N IH PA Athor M anscript N IH PA Athor M anscript We next turn to a specific analysis of U-shaped responses (figures 2d, 3a–c). In our experiments, relative values were generally stable within any recording session. However, the relative value of any given pair of juices could vary from day to day. For example, the relative value of apple juice versus peppermint tea varied between 1.5 and 3. This variability provides a further opportunity to test the neuronal encoding of value; specifically, U-shaped responses should reflect this variability. For this analysis, we test the entire neuronal population with the regression function a0 +aA (# A) +aB (# B), where (#A) and (#B) represent the amounts of juices A and B chosen by the monkey. We define a response to be U-shaped if both aA and aB differ from zero (p<0.01). If U-shaped responses indeed encode the value of the chosen juice, the slope ratio k*=aA/aB should be, for each response, equal to the relative value (n*) measured in that session. Most importantly, the slope ratio k* obtained for different responses should co-vary with n*. To test this prediction, we compute the regression k*=b0+b1 n* separately for every juice pair. Averaging across juice pairs, we obtain b0=−0.13 (±0.15) and b1=1.05 (±0.15) (±s.e.m.), consistent with the identity k*=n*. This result demonstrates that Ushaped responses do not reflect the quantity of any particular juice ingredient (e.g., sugar). Rather, they encode the value monkeys assign to the juice they choose to consume (Supplementary Results, p.S10–S12; Figures S12–S14). With respect to the timing of neuronal activation, the average neuronal activity peaks shortly after the offer, declines during the delay, is low before and during the eye movement, and has two secondary peaks at juice delivery (Supplementary Figure S3). To appreciate how the variables offer value, chosen value, and taste are represented in OFC over time, we analyze the activity of each cell in 50-ms, non-overlapping time bins. Figure 4 shows the number of cells encoding each of the three variables at different times. Remarkably, the time profile of different variables seems to reflect the mental processes the monkey likely undertakes during a trial. Shortly after the offer, when the monkey presumably assigns values to the two juices, neurons encoding the offer value (i.e., the value of one juice or the other) are most prevalent. Also during the delay, many neurons encode the chosen value (i.e., the value of the juice the monkey will eventually consume), even though the choice is still covert (because the go signal has not been given yet). Finally, after the monkey has indicated its choice, before and after juice delivery, many neurons encode the taste of the chosen juice. Conceptually, responses encoding the chosen value are particularly interesting because, in addition to being independent of the visuomotor contingencies of the task, they are also independent of the specifics of the good, namely juice type and juice amount. These responses encode economic value in a rather non-specific way. Further research is necessary to establish whether this result generalizes to other kinds of goods, such as non-comestible goods26. The interpretation of offer value responses is made more cautiously because, assuming linear value functions, the value of a given amount of juice is proportional to the juice quantity. “Value” is known to modulate the activity of neurons in several sensory and motor areas19– 25. For example, neurons in the lateral intraparietal area activate when monkeys plan a saccade towards a particular location of the visual field; their response is enhanced when the eye movement is associated with higher value20. On this basis, it has been proposed that parietal neurons encoding the value of all possible courses of action form a common path for decisionmaking, and that their activity is actually the subject of economic theory27. According to this “action-based” model, economic choice is fundamentally choice between actions. The fact that neurons in OFC encode the economic value of offered and chosen goods per se, and not as modulation of visuomotor processes, suggests an alternative “good-based” model, according to which economic choice is fundamentally choice between goods. In this view, choice is made between goods, and a suitable motor action is subsequently planned and executed. Several arguments seem to favor the good-based model. From a computational perspective, a modular design separating the mental operations of [choosing] and [moving] is more parsimonious28, Padoa-Schioppa and Assad Page 3 Nature. Author manuscript; available in PMC 2009 January 23. N IH PA Athor M anscript N IH PA Athor M anscript N IH PA Athor M anscript 29. In addition, values processed in OFC are logically sufficient for good-based choice. The action-based model would thus imply that, during economic choice, the nervous system operates in a computationally inefficient way, while undertaking all the processes needed to choose efficiently. Finally, a vast literature links choice in various domains to OFC. For example, human patients and monkeys with OFC lesions can present eating disorders and hyperorality6–8, abnormal risk-seeking and gambling behavior9,10, and impulsivity, altered personality and social behavior6,11. In contrast, parietal lesions typically result in visuospatial deficits such as hemi-neglect or Balint’s syndrome30. In conclusion, together with other lines of evidence, the present results support a good-based psychological model of economic choice behavior.